Image processing apparatus, and non-transitory computer readable medium for reducing undesirable effect during image capturing

ABSTRACT

An image processing apparatus includes a receiving unit that receives a first image that is captured with an object irradiated with light from a light irradiating unit and a second image that is captured with the object not irradiated with light from the light irradiating unit, and a generating unit that generates a composite image by adjusting brightness of a whole area of the second image such that brightness at a specific location in the first image is approximately equal to brightness at a corresponding specific location in the second image, and by replacing in the first image a pixel value of a pixel that is affected by light irradiation by the light irradiating unit with a pixel value of a pixel in the second image that has been adjusted in brightness.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2018-177197 filed Sep. 21, 2018.

BACKGROUND (i) Technical Field

The present disclosure relates to an image processing apparatus, and anon-transitory computer readable medium.

(ii) Related Art

Japanese Unexamined Patent Application Publication No. 2010-74693discloses a digital camera. The digital camera includes an electronicflash device that irradiates an object with flash light, an imagingdevice that consecutively acquires a first image that is captured withthe object not irradiated with flash light, and a second image that iscaptured with the object irradiated with flash light, an imagecorrelation detection circuit that detects an image portion of eachimage having no correlation by comparing the first image with the secondimage, and a correction image generating circuit that generates a thirdimage by extracting from the second image the image portion having nocorrelation, removes from the first image the image portion having nocorrelation, and embedding the image portion extracted from the secondimage into the corresponding portion of the first image from which theimage portion having no correlation has been removed.

SUMMARY

Aspects of non-limiting embodiments of the present disclosure relate toan image processing apparatus and a non-transitory computer readablemedium. The image processing apparatus acquires an image that iscaptured in a more reduced effect that is caused with a lightirradiating unit irradiating an object with light in comparison with animage of the object that is captured with the light irradiation unitirradiating the object with light at one time only.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided animage processing apparatus. The image processing apparatus includes areceiving unit that receives a first image that is captured with anobject irradiated with light from a light irradiating unit and a secondimage that is captured with the object not irradiated with light fromthe light irradiating unit, and a generating unit that generates acomposite image by adjusting brightness of a whole area of the secondimage such that brightness at a specific location in the first image isapproximately equal to brightness at a corresponding specific locationin the second image, and by replacing in the first image a pixel valueof a pixel that is affected by light irradiation by the lightirradiating unit with a pixel value of a pixel in the second image thathas been adjusted in brightness.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present disclosure will be described indetail based on the following figures, wherein:

FIG. 1A a perspective external view of an image processing apparatus ofan exemplary embodiment, and FIG. 1B is a perspective view illustratinghow an object is photographed using the image processing apparatus;

FIG. 2 illustrates the state of the object that is photographed with theobject not irradiated with light from a flashing device;

FIG. 3 illustrates the state of the object that is photographed with theobject irradiated with light from the flashing device;

FIG. 4 illustrates a hardware configuration of the image processingapparatus of the exemplary embodiment;

FIG. 5 is a functional block diagram of the image processing apparatusof FIG. 4;

FIG. 6 is a flowchart illustrating an image composing process performedby the image processing apparatus of the exemplary embodiment;

FIG. 7A illustrates the state of a first image that is obtained bycapturing an image of the object with the object irradiated with flashlight and then by projective-transforming the image, and FIG. 7Billustrates the state of a second image that is obtained by capturing animage of the object with the object not irradiated with the flash lightand then by projective-transforming the image;

FIGS. 8A through 8C illustrate the concept of the image composingprocess of the exemplary embodiment, wherein FIG. 8A illustratesluminance of a pixel row of the first image of FIG. 7A across lineVIIIA-VIIIA, FIG. 8B illustrates luminance of pixel rows of the firstand second images with brightness of the second image adjusted, and FIG.8C illustrates luminance of a pixel row of a composite image into whichthe first and second images are combined;

FIG. 9 illustrates the concept of an image processing system of amodification of the exemplary embodiment of the disclosure;

FIG. 10 is a functional block diagram illustrating a terminal device ofthe modification of the exemplary embodiment;

FIGS. 11A and 11B illustrate the configuration of an image processingserver of the image processing system of the modification of theexemplary embodiment, wherein FIG. 11A illustrates the hardwareconfiguration of the image processing server, and FIG. 11B illustratesthe functional block diagram of the image processing server;

FIG. 12 is a flowchart illustrating the process of the terminal deviceof the modification of the exemplary embodiment; and

FIG. 13 is a flowchart illustrating the process of the image processingserver of the modification of the exemplary embodiment.

DETAILED DESCRIPTION

An image processing apparatus 20 of an exemplary embodiment of thedisclosure is described with reference to FIGS. 1A and 1B. FIG. 1A is aperspective external view of the image processing apparatus 20 of theexemplary embodiment, and FIG. 1B is a perspective view illustrating howan object 30 is photographed using the image processing apparatus 20.Referring to FIGS. 1A and 1B, the image processing apparatus 20 of theexemplary embodiment is a tablet computer. The disclosure is not limitedto the tablet computer. For example, the image processing apparatus 20may be a smart phone or a digital camera as long as the configurationthereof is the one described below. The image processing apparatus 20may be a terminal device, such as a notebook computer with a camera. Ahousing 200 of the image processing apparatus 20 includes on the rearsurface thereof a camera 206, and a flash generating device 207. Thehousing 200 of the image processing apparatus 20 includes on the frontsurface thereof a display 204 that occupies most of the front surface.

As described below, when the object 30 is photographed, the video of theobject 30 captured by the camera 206 is displayed on the display 204 ona real-time basis. A user may photograph the object 30 by operating aninput interface (described below) mounted on the display 204 whileobserving the object 30 displayed on the display 204.

In accordance with the exemplary embodiment, the object 30 may be a testimage of a printer or a multi-function apparatus, output by an imageforming apparatus. The test image is an image printed on an entire papersheet with one or a mixture of black, cyan, magenta, and yellow colorsat a given density, or may be an entire half-tone image. The test imageis used to check the presence or absence of nonuniformity or bandingwhen image data at a specific gradation value is printed out. The objectthat is photographed in accordance with the exemplary embodiment of thedisclosure is not limited to the test image. The object 30 may be adifferent image.

If an entire image filled with the same color (solid image) isphotographed as an object by the image processing apparatus 20 with theobject not irradiated with light from the flash generating device 207, ashadow may be created on the object 30 by light incoming from theoutside as illustrated in FIG. 2. For example, nonuniformity of lightreflected from the object 30, such as luminance nonuniformity, may becaused by the shadow of the image processing apparatus 20 itself or theshadow of a hand of a photographer.

The object 30 may be photographed within close distance with the flashgenerating device 207 irradiating the object 30 with light in order tocapture an image that is free from the effect of the shadow caused bylight incoming from the outside. Referring to FIG. 3, concentricgradations about the center where light is projected appear on theobject 30. If such a problem occurs, it is difficult to check thepresence or absence of nonunformity or the presence or absence ofbanding when the image data at a fixed gradation value is printed out.It is desirable to reduce an effect involved in image capturing.

The effect caused by the flash generating device 207 radiating light isdesirably reduced in the image captured by the image processingapparatus 20 of the exemplary embodiment. The image is thus captured ina manner described below.

The configuration of the image processing apparatus 20 of the exemplaryembodiment is described with reference to FIG. 4. FIG. 4 illustrates thehardware configuration of the image processing apparatus 20 of theexemplary embodiment.

Referring to FIG. 4, the image processing apparatus 20 includes acontrolling microprocessor 201, a memory 202, a storage device 203, adisplay 204, an input interface 205, a camera 206, and a flashgenerating device 207. Each of these elements is connected to controlbus 208.

The controlling microprocessor 201 controls the operation of eachelement in the image processing apparatus 20 in accordance with acontrol program stored on the storage device 203.

An image of the object 30 photographed by the camera 206 and a compositeimage generated by an image generating unit are temporarily stored onthe memory 202.

The storage device 203 includes a solid-state drive (SSD) or a hard disk(HDD), and stores a control program that controls each element in theimage processing apparatus 20.

The display 204 includes a liquid-crystal display or an organicelectroluminescent (EL) display, mounted on the housing 200 of the imageprocessing apparatus 20, and displays information processed by a displaycontroller described below.

The input interface 205 is a touch panel overlaid on the front surfaceof the display 204, and serves as an input unit on which a useroperating the image processing apparatus 20 inputs instructions.

The camera 206 is arranged on the rear surface of the housing 200 of theimage processing apparatus 20. The camera 206 captures the image of theobject 30 in response to an instruction that the user inputs byoperating the input interface 205, and an instruction from a cameracontroller described below. The captured image is stored on the memory202.

The flash generating device 207 includes a light-emitting diode (LED)light. The flash generating device 207 serves as a light irradiationunit that irradiates the object 30 with flash light when the object 30is photographed in response to the instruction from the cameracontroller.

The functionality of the image processing apparatus 20 of the exemplaryembodiment is described with reference to FIG. 5. FIG. 5 is a functionalblock diagram illustrating the image processing apparatus 20 of FIG. 4.Referring to FIG. 5, the image processing apparatus 20 performs thefunctionalities of a camera controller 211, an image receiving unit 212,an image generating unit 213, and a display controller 214 when thecontrolling microprocessor 201 executes the control program stored onthe storage device 203.

The camera controller 211 controls the operations of the camera 206 andthe flash generating device 207. If the object 30 in the video capturedby the camera 206 satisfies a specific condition, for example, if theobject 30 is recognized in a predetermined size, the camera controller211 adjusts focus. The camera controller 211 captures a first image bycausing the flash generating device 207 to irradiate the object 30 withflash light, and immediately in succession to the capturing of the firstimage, captures a second image by not causing the flash generatingdevice 207 to irradiate the object 30 with flash light, and stores thefirst and second images on the memory 202.

The image receiving unit 212 receives the first image that has beencaptured with the flash generating device 207 irradiating the object 30with flash light and the second image that has been captured with theflash generating device 207 not irradiating the object 30 with flashlight. In other words, the image receiving unit 212 retrieves the firstimage and the second image from the memory 202.

The image generating unit 213 performs projective transformation suchthat the first and second images retrieved by the image receiving unit212 are corrected to be in a specified size. More specifically, when theobject 30 is photographed by the camera 206 in the image processingapparatus 20, the image of the object 30 is captured at a slight slantangle. By performing the projective transformation on the capturedimages, distortion is removed from the images and the images areadjusted in size.

The image generating unit 213 adjusts brightness of the whole secondimage such that brightness at a specific location in the first imageretrieved and projective-transformed by the image receiving unit 212 isapproximately equal to brightness at a specific location in the secondimage retrieved and projective-transformed by the image receiving unit212. The image generating unit 213 generates a composite image byreplacing a pixel value of a pixel that has been affected by theirradiation of flash light from the flash generating device 207 with apixel value of a pixel in the brightness-adjusted second image.

More specifically, the image generating unit 213 generates the compositeimage by replacing in the first image a pixel value of a pixel higher inbrightness than a pixel in the second image adjusted in brightness, as apixel that has been affected by the flash light irradiation by the flashgenerating device 207, with the pixel value of the pixel in the secondimage adjusted in brightness. Alternatively, the image generating unit213 generates the composite image by selecting a pixel value of thefirst image retrieved by the image receiving unit 212 or a pixel valueof the second image retrieved by the image receiving unit 212, whicheveris lower.

The display controller 214 processes an image to be displayed on thedisplay 204. The display controller 214 performs control to display onthe display 204 a video that is captured by the camera 206 when anobject is photographed by the camera 206, to display on the display 204an image generated by the image generating unit 213, or to display onthe display 204 a user interface (UI) that allows a user to input avariety of instructions.

The image processing apparatus 20 of the exemplary embodiment isdescribed with reference to FIGS. 6, 7A and 7B, and 8A through 8C. FIG.6 is a flowchart illustrating an image composing process performed bythe image processing apparatus 20 of the exemplary embodiment. FIGS. 7Aand 7B illustrate images that have undergone projective transformation.Specifically, FIG. 7A illustrates the first image that is captured withthe object irradiated with flash light, and FIG. 7B illustrates thesecond image that is captured with the object not irradiated with theflash light. FIGS. 8A through 8C illustrate the concept of the imagecomposing process of the exemplary embodiment. FIG. 8A illustratesluminance of a pixel row of the image captured with the objectirradiated with the flash light from the flash generating device 207 andluminance of a pixel row of the image captured with the object notirradiated with the flash light from the flash generating device 207.FIG. 8B illustrates the concept of corrected luminance. FIG. 8Cillustrates luminance of the pixel row of the composite image.

In step S401, the camera controller 211 causes the camera 206 to startoperating in response to an instruction from a user. The displaycontroller 214 displays a video captured by the camera 206 on thedisplay 204. The camera controller 211 determines whether the object 30to be photographed is recognized as satisfying a predeterminedcondition. The condition is related to whether the object 30 isrecognized as being in a predetermined size or larger. If thepredetermined condition is not satisfied, an attempt to recognize theobject 30 continues with the camera 206 inactive. If the object 30 thatsatisfies the predetermined condition is not recognized even after thetime elapse of a predetermined period of time, the image composingprocess may end.

If the object 30 to be photographed is recognized as satisfying thepredetermined condition, the process proceeds to step S402. The cameracontroller 211 causes the flash generating device 207 to irradiate theobject 30 with flash light for image capturing.

In step S403, the camera controller 211 stores on the memory 202, as thefirst image, an image that is captured with the flash generating device207 irradiating the object 30 with the flash light.

In step S404, the camera controller 211 immediately captures the imageof the object 30 with the flash generating device 207 not irradiatingthe object 30 with the flash light.

In step S405, the camera controller 211 stores on the memory 202, as thesecond image, an image that is captured with the flash generating device207 not irradiating the object 30 with the flash light.

In step S406, the image receiving unit 212 retrieves the first andsecond images from the memory 202. The image generating unit 213projective-transforms each of the first and second images, and correctsthe projective-transformed first and second images to be equal to eachother in size. During the projective transformation, distortion in thefirst and second image is removed. The intended image size during theprojective transformation is pre-set by the user. Alternatively, duringthe projective transformation, the display controller 214 may requestthe user to specify the image size, and the user may operate the inputinterface 205 to specify the image size to which the images have to bescaled.

In step S407, the image generating unit 213 adjusts the brightness ofthe whole second image such that brightness at specific locations in thefirst image projective-transformed may be approximately equal tobrightness at specific locations in the second imageprojective-transformed. The specific locations are multiple locationsoff the central region of each of the first image and second imagesprojective-transformed. For example, if four imaginary lines are drawnfrom the center of each of the first image and second imagesprojective-transformed to the four corners thereof, the specificlocations are four points, each point at one-third length point alongeach imaginary line from the center of each image. The specificlocations may be other multiple points in the images. The specificlocations are not limited to four points, and may be a different numberof points.

The brightness may be any index indicating a luminance value, agradation value, lightness, or a degree of brightness of an image. Inthe following discussion, the brightness is luminance. According to theexemplary embodiment, the image generating unit 213 adjusts theluminance value of the whole area of the second image that has beenprojective-transformed such that the mean of the luminance values at thefour locations in the first image projective-transformed and the mean ofthe luminance values at the four locations in the second imageprojective-transformed are approximately equal to each other,specifically within a predetermined range, or more preferably convergeto the same value. Besides the mean of the luminance values, a varianceof luminance or contrast may be calculated, and the second image may beadjusted such that the variance falls within a predetermined range.

The operation in step S407 is described with reference to FIGS. 7A and7B, and FIGS. 8A through 8C. FIG. 7A illustrates the state of the firstimage that is obtained by capturing the image of the object 30 with theobject 30 irradiated with flash light and then byprojective-transforming the image. FIG. 7B illustrates the state of thesecond image that is obtained by capturing an image of the object 30with the object 30 not irradiated with flash light and then byprojective-transforming the image. FIG. 8A illustrates luminance of thepixel row of the first image of FIG. 7A along line VIIIA-VIIIA andluminance of the pixel row of the second image of FIG. 7B along lineVIIIA-VIIIA. FIG. 8B illustrates luminance of pixel rows of the firstand second images with brightness of the second image adjusted, and FIG.8C illustrates luminance of the pixel row of the composite image intowhich the first and second images are combined.

Referring to FIGS. 7A and 7B, and FIGS. 8A through 8C, the horizontalaxis of FIGS. 8A through 8C illustrating the luminance of the pixel rowsis an X coordinate when the image is cut in a horizontal direction, andthe vertical axis represents luminance values. Referring to FIG. 8A,luminance of the image captured with the flash generating device 207irradiating the object with the flash light is typically higher thanluminance of the image captured with the flash generating device 207 notirradiating the object with the flash light. Furthermore, the plot ofthe luminance of the image captured with the flash generating device 207irradiating the object with the flash light has a hill-like shape alongthe horizontal axis with the peak point thereof at the central regionand the slopes on both sides. Referring to FIG. 8B, the image generatingunit 213 adjusts the luminance of the whole second imageprojective-transformed such that luminance values at points P in thefirst and second images projective-transformed are equal to each other(in other words, luminance values are generally increased as illustratedin FIG. 8B).

In step S408 of FIG. 6, the image generating unit 213 replaces the pixelvalue of a pixel that is affected by the flash light radiated by theflash generating device 207 with the pixel value of a pixel in thesecond image that has been adjusted in step S407. More specifically, theimage generating unit 213 replaces the pixel value of a pixel brighterin the first image than in the second image that has been adjusted inbrightness, as a pixel that has been affected by the flash lightirradiated by the flash generating device 207, with a pixel value of apixel in the second image that has been adjusted in brightness.Alternatively, the image generating unit 213 generates a composite imageby comparing the projective-transformed first image with thebright-adjusted second image on a per pixel basis, and selecting a pixelvalue, whichever is less bright.

FIG. 8C illustrates the composite image as the resulting state. Within arange defined by specific points P, pixels in the second image adjustedin brightness are used as pixels in the composite image of FIG. 8C.Outside the range defined by the specific points P, the pixels in thefirst image projective-transformed are used as pixels in the compositeimage of FIG. 8C.

In step S409 of FIG. 6, the image generating unit 213 stores thecomposite image generated in step S408 on the memory 202. The displaycontroller 214 displays the composite image on the display 204 such thatthe user may observe the composite image. The image composing processthen ends.

In the image composing process described above, in the first imageprojective-transformed, the pixels brighter than those in the secondimage adjusted in brightness are determined to be the pixels that areaffected by the flash light radiated by the flash generating device 207,and the pixel values of those pixels are thus replaced with the pixelvalues in the second image adjusted in brightness. The disclosure is notlimited to the process. The pixel values of pixels in a region servingas a target to be replaced and of pixels surrounding the region arereplaced with values that are calculated using pixel values in the firstimage projective-transformed and pixel values in the second pixeladjusted in brightness. In such a case, the pixel value of the pixelserving as a replacement may be obtained by combining, using SoftMinimumfunction, a pixel value in the first image projective-transformed and apixel value in the second image adjusted in brightness.

According to the exemplary embodiment described above, the second imageis captured with the object not irradiated with the flash lightimmediately after the first image is captured with the object irradiatedwith the flash light. Conversely, the first image may be captured withthe object irradiated with the flash light immediately after the secondimage is captured with the object not irradiated with the flash light.

Modification of the exemplary embodiment of the disclosure is describedwith reference to FIGS. 9 through 13. FIG. 9 illustrates the concept ofan image processing system 60A of a modification of the exemplaryembodiment of the disclosure. FIG. 10 is a functional block diagramillustrating a terminal device 20A of the modification. FIGS. 11A and11B illustrate the configuration of an image processing server 80A ofthe image processing system 20A of the modification. FIG. 11Aillustrates the hardware configuration of the image processing server80A, and FIG. 11B is the functional block diagram of the imageprocessing server 80A. FIG. 12 is a flowchart illustrating the processof the terminal device 20A of the modification. FIG. 13 is a flowchartillustrating the process of the image processing server 80A of themodification.

In the exemplary embodiment described with reference to FIG. 1 throughFIG. 8, the photographing of the object 30 and the composing process ofthe images are performed by the single image processing apparatus 20. Inthe modification described with reference to FIG. 9 through FIG. 13, thephotographing of the object 30 and the displaying of the composite imagesubsequent to image processing are performed on the terminal device 20A,and the image processing is performed on the image processing server80A.

Referring to FIG. 9, the image processing system 60A of the modificationincludes the terminal device 20A and the image processing server 80A.The terminal device 20A and the image processing server 80A arerespectively connected to a network 70A. In the modification, theterminal device 20A is generally identical in hardware configuration tothe image processing apparatus 20 except that the terminal device 20Aincludes a communication interface (not illustrated). Elements identicalto those in the image processing apparatus 20 are designated with thesame reference symbols and the detailed discussion is omitted herein.

Referring to FIG. 10, a controlling microprocessor 201 in the terminaldevice 20A of the modification performs a control program stored on thestorage device 203. The terminal device 20A thus has functionalities ofa camera controller 211A, a display controller 212A, and an imagetransceiver unit 213A. The camera controller 211A and the displaycontroller 212A are respectively identical in functionality to thecamera controller 211 and the display controller 214 in the imageprocessing apparatus 20 of the exemplary embodiment, and the detaileddiscussion thereof is omitted herein.

The first image is captured with the flash generating device 207irradiating the object and the second image is captured with the flashgenerating device 207 not irradiating the object. The first image andthe second image are stored on the memory 202 or the storage device 203.In response to an instruction input by the user operating the inputinterface 205, the image transceiver unit 213A transmits the first imageand the second image to the image processing server 80A via the network70A or receives the composite image produced on the image processingserver 80A from the image processing server 80A via the network 70A.

Referring to FIG. 11A, the image processing server 80A of themodification includes a controlling processor 801A, a memory 802A, astorage device 803A, and a communication interface 804A, each thereofconnected to a control bus 805A.

The controlling processor 801A controls the operation of each element inthe image processing server 80A in accordance with the control programstored on the storage device 803A. The first image captured with theflash light radiating and the second image with the flash light notradiating are temporarily stored on the memory 802A. The storage device803A includes a hard disk (HDD) and/or a solid-state drive (SDD), andstores the control program that controls each element in the imageprocessing server 80A. The communication interface 804A controlscommunication that the image processing server 80A performs with theterminal device 20A via the network 70A.

Referring to FIG. 11B, the controlling processor 801A in the imageprocessing server 80A executes the control program stored on the storagedevice 803A. The image processing server 80A has thus thefunctionalities of an image receiving unit 811A, an image generatingunit 812A, and an image transmitting unit 813A.

The image receiving unit 811A receives from the terminal device 20A thefirst image captured with the flash light radiating and the second imagewith the flash light not radiating, and temporarily stores the first andsecond images on the memory 802A.

The image generating unit 812A adjusts the brightness of the wholesecond image such that the brightness at a specific location in thefirst image acquired by the image receiving unit 811A is approximatelyequal to the brightness at a corresponding specific location in thesecond image acquired by the image receiving unit 811A. The imagegenerating unit 812A generates a composite image by replacing in thefirst image the pixel value of a pixel that is affected by theirradiation of the flash light with the pixel value of a pixel in thesecond image that is adjusted in brightness. The process described aboveremains unchanged from the process of the image generating unit 213 inthe image processing apparatus 20 of the exemplary embodiment, and thedetailed discussion thereof is omitted herein.

The image transmitting unit 813A transmits to the terminal device 20Athe composite image generated by the image generating unit 812A via thecommunication interface 804A and the network 70A.

The process of the image processing system 60A of the modification isdescribed below. The process of the terminal device 20A is describedfirst with reference to FIG. 12. In step S901 of FIG. 12, the cameracontroller 211A causes the camera 206 to start operating in response toan instruction that is input by the user operating the terminal device20A. The terminal device 20A photographs the object by causing the flashgenerating device 207 to irradiate the object with the flash light, andtemporarily stores the captured image as the first image on the memory202.

In step S902, the image transceiver unit 213A transmits the first imageto the image processing server 80A via the network 70A.

In step S903, the camera controller 211 photographs the object with theflash generating device 207 not irradiating the object, and stores thecaptured image as the second image on the memory 202.

In step S904, the image transceiver unit 213A transmits the second imageto the image processing server 80A via the network 70A. In themodification of the exemplary embodiment, the first image and the secondimage are transmitted to the image processing server 80A each time eachof the first and second images is captured. Alternatively, the first andsecond images, after being captured, may be together transmitted to theimage processing server 80A.

Referring to FIG. 13, the composing process of the images is performedon the image processing server 80A. In step S905, the image transceiverunit 213A receives the composite image from the image processing server80A, and stores the composite image on the memory 202.

In step S906, the display controller 212A displays on the display 204the composite image received in step S905 such that the user may viewthe composite image. The process thus ends.

The process of the image processing server 80A is described withreference to FIG. 13. In step S1001 of FIG. 13, the image receiving unit811A in the image processing server 80A receives the first image and thesecond image from the terminal device 20A. The first image and thesecond image may be received together at the same time or separatelywith a time difference therebetween. The first image and the secondimage are temporarily stored on the memory 802A.

In step S1002, the image generating unit 812A projective-transforms eachof the first and second images such that the two images are equal insize. The image size serving as a target of the projectivetransformation may be preset by the user, and the information on theimage size may be received together with the first and second images.Alternatively, the terminal device 20A may be requested to specify theimage size, and the user may specify the image size by operating theinput interface 205 in the terminal device 20A. The information on thespecified image size may be transmitted to the image processing server80A.

In step S1003, the image generating unit 812A adjusts the brightness ofthe whole second image such that the brightness at a specific locationin the first image projective-transformed is approximately equal to thebrightness at a specific location in the second imageprojective-transformed. The operation is generally identical to theoperation that the image generating unit 213 in the image processingapparatus 20 performs in step S407 of FIG. 6, and the detaileddiscussion thereof is omitted herein.

In step S1004, the image generating unit 812A generates a compositeimage by replacing in the first image projective-transformed the pixelvalue of a pixel that is affected by the irradiation of the flash lightwith the pixel value of a pixel in the second image that is adjusted inbrightness in step S1003. The operation is generally identical to theoperation that the image generating unit 213 in the image processingapparatus 20 of the exemplary embodiment performs in step S408 of FIG.6, and the detailed discussion thereof is omitted herein.

In step S1005, the image transmitting unit 813A transmits the compositeimage generated in step S1004 to the terminal device 20A via thecommunication interface 804A and the network 70A. The process thus ends.

In the modification of the exemplary embodiment, the terminal device 20Acaptures the first image and the second image, and successivelytransmits the first image and the second image to generate the compositeimage. The disclosure is not limited to this method. The first image andthe second image may be first stored on the storage device 203 in theterminal device 20A, and the image processing server 80A may laterretrieve the first and second images, and generate the composite image.

The foregoing description of the exemplary embodiments of the presentdisclosure has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thedisclosure to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments was chosen and described in order to best explain theprinciples of the disclosure and its practical applications, therebyenabling others skilled in the art to understand the disclosure forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of thedisclosure be defined by the following claims and their equivalents.

What is claimed is:
 1. An image processing apparatus comprising aprocessor configured to: receive a first image that is captured with anobject irradiated with light from a light irradiating unit and a secondimage that is captured with the object not irradiated with light fromthe light irradiating unit wherein the first image and the second imageare consecutively captured; adjust brightness of a whole area of thesecond image based on brightness of the first image to generate anadjusted second image, wherein a difference between brightness at aspecific location in the first image and brightness at a correspondingspecific location in the adjusted second image is within a predeterminedrange, and replace in the first image a pixel value of a pixel that hashigher brightness than a pixel in the adjusted second image with a pixelvalue of the pixel in the adjusted second image to generate a compositeimage.
 2. The image processing apparatus according to claim 1, whereinthe processor generates the composite image by selecting the pixel valueof the pixel in the first image or the pixel value of the pixel in theadjusted second image, whichever is lower in brightness.
 3. The imageprocessing apparatus according to claim 2, wherein the processor adjuststhe brightness of the whole area of the second image such that a meanvalue of brightness at a plurality of specific locations in the firstimage is approximately equal to a mean value of brightness at aplurality of corresponding specific locations in the adjusted secondimage.
 4. The image processing apparatus according to claim 3, whereineach of the specific locations in the first image is off a centralregion of the first image and each of the specific locations in theadjusted second image is off a central region of the adjusted secondimage.
 5. The image processing apparatus according to claim 2, furthercomprising: an imaging unit that captures an image of the object; andthe light irradiating unit that irradiates the object with light whenthe imaging unit captures the image of the object.
 6. The imageprocessing apparatus according to claim 1, wherein the processorgenerates the composite image by replacing in the first image a pixelvalue of a pixel in a target region to be replaced and a pixel value ofa pixel in a region surrounding the target region with a value that iscalculated using the pixel value of the pixel in the first image and thepixel value of the pixel in the adjusted second image.
 7. The imageprocessing apparatus according to claim 6, wherein the processor adjuststhe brightness of the whole area of the second image such that a meanvalue of brightness at a plurality of specific locations in the firstimage is approximately equal to a mean value of brightness at aplurality of corresponding specific locations in the adjusted secondimage.
 8. The image processing apparatus according to claim 7, whereineach of the specific locations in the first image is off a centralregion of the first image and each of the specific locations in theadjusted second image is off a central region of the adjusted secondimage.
 9. The image processing apparatus according to claim 6, furthercomprising: an imaging unit that captures an image of the object; andthe light irradiating unit that irradiates the object with light whenthe imaging unit captures the image of the object.
 10. The imageprocessing apparatus according to claim 1, wherein the processor adjuststhe brightness of the whole area of the second image such that a meanvalue of brightness at a plurality of specific locations in the firstimage is approximately equal to a mean value of brightness at aplurality of corresponding specific locations in the adjusted secondimage.
 11. The image processing apparatus according to claim 10, whereineach of the specific locations in the first image is off a centralregion of the first image and each of the specific locations in theadjusted second image is off a central region of the adjusted secondimage.
 12. The image processing apparatus according to claim 1, whereinthe processor adjusts the brightness of the whole area of the secondimage such that a mean value of brightness at a plurality of specificlocations in the first image is approximately equal to a mean value ofbrightness at a plurality of corresponding specific locations in theadjusted second image.
 13. The image processing apparatus according toclaim 12, wherein each of the specific locations in the first image isoff a central region of the first image and each of the specificlocations in the adjusted second image is off a central region of theadjusted second image.
 14. The image processing apparatus according toclaim 1, further comprising: an imaging unit that captures an image ofthe object; and the light irradiating unit that irradiates the objectwith light when the imaging unit captures the image of the object. 15.The image processing apparatus according to claim 14, wherein theimaging unit captures the image with the object not irradiated withlight from the light irradiating unit after the imaging unit capturesthe image with the object irradiated with light from the lightirradiating unit.
 16. The image processing apparatus according to claim1, further comprising: an imaging unit that captures an image of theobject; and the light irradiating unit that irradiates the object withlight when the imaging unit captures the image of the object.
 17. Theimage processing apparatus according to claim 16, wherein the imagingunit captures the image with the object not irradiated with light fromthe light irradiating unit after the imaging unit captures the imagewith the object irradiated with light from the light irradiating unit.18. A non-transitory computer readable medium storing a program causinga computer to execute a process for processing an image, the processcomprising: receiving a first image that is captured with an objectirradiated with light and a second image that is captured with theobject not irradiated with light, wherein the first image and the secondimage are consecutively captured; adjusting brightness of a whole areaof the second image based on brightness of the first image to generatean adjusted second image, wherein a difference between brightness at aspecific location in the first image and brightness at a correspondingspecific location in the adjusted second image is within a predeterminedrange, and replacing in the first image a pixel value of a pixel thathas higher brightness than a pixel in the adjusted second image with apixel value of the pixel in the adjusted second image to generate acomposite image.
 19. A non-transitory computer readable medium storing aprogram causing a computer to execute a process for processing an image,the process comprising: capturing a second image with an object notirradiated with light after a first image is captured with the objectirradiated with light, wherein the first image and the second image areconsecutively captured; and adjusting brightness of a whole area of thesecond image based on brightness of the first image to generate anadjusted second image, wherein a difference between brightness at aspecific location in the first image and brightness at a correspondingspecific location in the adjusted second image is within a predeterminedrange, and replacing in the first image a pixel value of a pixel thathas higher brightness than a pixel in the adjusted second image with apixel value of the pixel in the adjusted second image to generate acomposite image.